CVPR 2009 Program

Main CVPR2009 Paper Acceptance Decisions. 61 papers are accepted for ORALPresentation (4.1%). 322 papers are accepted for POSTER Presentation (22.1%).

Here are some interesting stats for CVPR 2009 (Click on the Plots to see bigger images, or Download DATA and do your own plotting!).

Plot of Paper Acceptance by Area (Orals, Posters, and Total) (DATA below)

CVPR Acceptance Rates from 1985 to 2009. (Data below.)

• CVPR 2009 Acceptances by Area Data
• CVPR 1985 to 2009 Data

Once you’ve been notified that the papers have been assigned to you, please log in to the site and follow these steps:

1. Download papers and check for possible conflict or submission rule violation:

• In the “Area Chair” console, click on “Select Reviewer Candidates”.
• You can download all the papers in a ZIP file from “Download All Files”. To reduce bandwidth, we exclude the supplementary files from the ZIP file.
• Contact the program chairs immediately (cvpr09-pc-chairs@googlegroups.com) if: 
  1. You think you are in any way conflicted with the paper.
  2. There is a violation of the stated paper submission rules. 
  • Such a violation include:
    • Not anonymous (names listed on front page),
    • Over 8 pages, 
    • Is double submission,
    • Supplementary material includes newer version of the paper.
  • Please specify the exact nature of the violation.

2. Familiarize yourself with “Select Reviewer Candidates” page (part of it shown below):

• “Paper Summary” label: next to it, you’ll see the icons ”+” and ”-”. Clicking on ”+” shows you all the abstracts; clicking on “-” collapses all them back.
• At the end of each paper title, you’ll see “+” as well. This has the same function of showing the abstract for that paper, toggling to “-” at the same time, which collapses it when selected.
• If you prefer to download the supplementary files, you’ll have to download them on a per-paper basis (under “Download Files”).

3. Add reviewer suggestions. For each paper, click on “Change Suggestions”. In the “Edit Candidates” page (part of it shown below):

• The reviewers are initially listed based on “Reviewer Relevance”. This number is computed based on matches on Primary and Secondary Subject Areas.
• Please take the time to familiarize yourself with the table entries:
  * Left-most column allows you to add the current reviewer to your list of suggestions.
  * Pick at least 5 people, making sure not more than two people are from the same organization. This will increase your chances of getting your selections. Note that we list ALL reviewers, regardless of whether there’s conflict with the paper. We do this to discourage speculation on authorship.
  * Please bear in mind that “Researcher/Faculty” has a limit of 12 papers to review while “Graduate Student” has a limit of 5.
  * Clicking on any of the column heading sorts according to its description.
• You can filter the reviewers. Here are two examples:
  
  * Filter example 1 (you are only interested in looking at reviewers whose relevance is at least 0.25): Click on “Add Filter Condition”, from left to right, select “Relevance” and “>=”, and type in “0.25″. Then click on “Find Reviewers”.
  * Filter example 2 (you are only interested in reviewers who are “Researcher/Faculty”): Click on “Add Filter Condition”, from left to right, select “Reviewer Type”, “=”, and “Researcher/Faculty”. Then click on “Find Reviewers”.
  * You can pipe multiple filters together (say, adding the first condition, then adding the second condition using “Add Filter Condition”).
  * If you have mistakenly clicked on “Add Filter Condition” and created a placeholder for a filter condition, click on the leftmost “X” icon to remove it.
• The higher the reviewer is ranked, the higher the weight assigned to him/her during the paper-to-reviewer assignment process. You can re-order a reviewer by clicking on “Move Up” or “Move Down” next it the name.
• Don’t forget to click on the “Save Changes” button to submit any changes you’ve made.

4. (Optional) Add a new reviewer (in “Edit Candidates” page):

• This should be done sparingly. There is usually a good reason why the person you are looking for is not in the reviewer list.
• Double-check your spelling when doing your search to make sure that person is really not in the list.
• Click on the “Add New Reviewer” button (arrowed above). Please read the instructions in the ”Add New Reviewer” page. In this page, you will be asked to first supply the email of the new reviewer. The system checks if that person is already a reviewer based on the email address.
• If that person is already a reviewer, you will be asked to go back to the previous page to add this person through the reviewer list.
• If that person already has an account but is not a reviewer, you need to supply reviewer type, conflict domain(s), and subject area(s). If that person does not have an account, you will need to supply the contact information as well. We assume you must know this person reasonably well to specifically request for him/her. (CMT will issue the invitation anonymously on your behalf. If that person later agrees to review, he/she will have the opportunity to edit reviewer information.)
• Once you have completed the request, that person will be added to the reviewer list. You will need to return to the reviewer list and add this new reviewer in.
• We will follow up with the newly added reviewers to ensure that they agree within a few days of the request, as we cannot allow too long of a period for such reviewer decisions to be made. We will notify you if any invitation to review is declined (you will then have to select someone else).
• We will be disabling the “Add New Reviewer” feature after Dec. 8 5PM PST (Dec. 9 1AM GMT).

5. Still need help?

During the reviewing phase (Nov 22, 2009 - Feb 30, 2009), please periodically log in to the CMT site and monitor the review progress.

Phases:

1. Dec 1 - Dec 15, 2008: Phase 1: Assigning Reviewers. 
2. Jan 1 - Jan 29, 2009: Phase 2: Check Incoming reviews, follow up with reviewers.
3. Jan 29 - Feb 5, 2009: Phase 3: Initiate discussions with reviewers, Write Draft Consolidation Reports.
4. Feb 6 - Feb 13, 2009: Phase 4: Rebuttal Period. Grade Reviewers
5. Feb 13 - Feb 17, 2009: Phase 5: Review Author Feedback and Discuss with Reviewers
6. Feb 13 - Feb 20, 2009: Phase 6: Prepare for AC Meeting. Discuss with AC buddies, Propose decisions on paper. 
7. Feb 21 - Fen 22, 2009: Phase 7: AC Meeting
8. NEED HELP

Phase 2. Jan 1 to review deadline (5PM PST Jan. 29, 2009, Thursday): Check incoming reviews

• In “Area Chair” console, select “Consolidation Reports”.
• You should be able to see all the reviewers’ names and the review status right next to each. For this incarnation of CMT, we just highlight cases where reviews have not been submitted (in red).
• For each paper, you can:
  * View all the reviews (”View All”).
  * Grade the reviews (”Rate Reviewers”).
  * Email the reviewers anonymously via CMT (”Email Reviewers”). You can select all the assigned reviewers for that paper (default), or just those not done with reviewing, or specific reviewer(s). For the last item, you will be able to edit the recipient list. Remember: Don’t identify yourself or the other reviewers in the email!
• You can also choose to bulk email all reviewers or all reviewers who have not completed their reviews (for all the papers in your stack). See below the title “Consolidation Reports” (”Email Reviewer”). Mouse over the “Email Reviewer” box and you will see the two options (mentioned earlier) from which to choose.
• You can view all reviews for all papers in a single webpage. Just select “View All Reviews” in the line right below the title “Consolidation Reports”.
• Important note on emailing via CMT: Since any email sent on your behalf is anonymously sent through CMT, the communication is one way only. Please tell your email recipients to not respond to the email, because there is no automatic routing of email to you; responses go directly to the CMT admin people. The CMT folks are handling a number of conferences at any given time and will not have the resources to forward such emails. So, be very specific about what you would like the reviewer(s) to do.
• It is the area chair’s responsibility to contact and urge the reviewers to finish in a timely manner.

Phase 3. Between review deadline and 5PM PST Feb. 5, 2009, Thursday: Initiate Discussions with Reviewers.

Right after the review deadline, please look through ALL the reviews. Please let us know who have yet to finish their reviews, and we will contact them directly.

If any review appears unreasonable (e.g., one-liner comments or overly harsh), please contact the reviewer and ask for clarification. Remember that communication is only one-way, so be specific about what you need from the reviewer. Check again in one or two days to see if there is any update to that review. Since we will not be keeping track of your emails (sorry!), please do the book-keeping yourself (best to make a note of what you’ve done).

3.1 When to Initiate a Discussion

We urge you to start a discussion any time you feel that the overall scores and/or the comments are insufficient to allow you to make a decision (e.g., all “Borderline”s). At the very least, initiate a discussion if the difference between the maximum and minimum scores is greater than 2 levels.

3.2 During the Discussion

You play the role of a moderator in the discussion. While it is ideal to reach a consensus, don’t feel obligated to force one—the discussions are mainly to allow you to make more informed decisions. It is ok if the reviewers end up not changing their overall scores as long as you’ve gained a better understanding of how they arrived at their scores.

Stick to the facts, and do not influence the outcome of the discussion by imposing your views on the paper. However, if you disagree with the majority view, say so, provide compelling reasons, and solicit feedback from the reviewers.

If the discussion looks like it is getting out of hand (e.g., when a reviewer makes inappropriate remarks or a battle of words erupts), please contact us (cvpr09-pc-chairs@googlegroups.com) immediately. Don’t forget to let us know the paper number.

For cases of “Borderline”s, if the discussion appears to not head towards any decision, we suggest you ask the question “If you had to choose now, would you accept or reject this paper?”

3.3 Anonymous Discussion Feature

This feature can be accessed from the “Consolidation Reports” page:

• You must do the following steps to enable the reviewers to participate in the discussion. Emailing the reviewers via CMT is NOT the right step to initiate the discussion. Reviewers can already see the other reviews for their papers.
• To initiate a discussion for a specific paper, select “View/Post Message” under the “Consolidation Report” column for that paper.
• In the “Paper Discussion” page, click on “Start A New Topic” to initiate the discussion (unless a reviewer took the initiative of starting one, in which case you reply; see next bullet item). We suggest that your first post starts with a very brief summary of the reviews, a request for them to look at the other reviews for details, followed by specific things you want the reviewers to address.
• Once a discussion has been initiated, click on “Reply” on the far right to continue posting on the same discussion thread.
• Please sign your posts as “Area Chair”, and identify the reviewers by the review number. Never identify yourself or other reviewers by name.
• Anytime a post is made (either by the area chair or a reviewer), the area chair and reviewers will receive an email notification from CMT with the subject that looks like “CVPR2009: New reviewer discussion posted for Paper ID XXX”. There is a link in the email you can use to join the discussion (after logging in, you will routed directly to the discussion page). Alternatively, you can just log in to CMT as usual.
• After you’ve posted, DO NOT REFRESH PAGE (e.g., by hitting F5)! This will generate another post with the exact same message!
• In the discussion, you (as AC) can see the identities of the posters. However, each reviewer will NOT know the identities of the other posters.
• NEW: The author rebuttal period is Feb 6, 2009 5pm PST - Feb 13, 2009 5pm PST. Once the author rebuttal period is over, you and the reviewers will be able to see the author rebuttal (but not before). We will be enabling discussions for a week past the rebuttal deadline, just in case reviewers have any reactions to the author rebuttals.
• Because of the frank nature of the discussions, the authors will not see them at any time. The consolidation reports will be visible to authors only after the AC meeting, when decisions are made known.
• Reviewer instructions for the discussion feature are given in Section 7 here. Note that reviewers can revise their reviews, up til the 5PM PST Feb 5 deadline.

Phase 4. Feb. 6, 2009 - Feb. 13, 2009: Rebuttal Period

Remember: Feb. 6-13 is the author rebuttal period. Reviewers are no longer allowed to modify their reviews after 5PM PST Feb. 5. Authors’ reviews will be visible to both reviewers and AC after Feb 13, 5pm EST. 

4.1 Grading Reviews

ACs are expected to grade each review. This information will be used in part to help us identify exceptional reviewers for recognition during the main CVPR09 conference in Miami. Some guidelines:

• “Highly Relevant”: Reviewer is on the ball on practically everything. The score is well justified, references are given to support claims that references are deficient and/or work is not original, and the reviewer appears very knowledgeable on the topic. You feel you can totally rely on this assessment.
• “Sufficient”: The review quality is mixed, but overall reliable. There are parts of the review that appear perfectly reasonable, but there are other parts that seem a bit deficient in some ways. You feel you are satisfied with the review and the reviewer has given you an evaluation you can use to make judgement on the paper.
• “Below Average”: Reviewer does not justify scores well, makes overly general unhelpful statements, states incorrect or misleading claims, provides very terse or irrelevant remarks, or appears to be overly biased. Basically, you feel you can ignore most of this review.

Phase 5. Feb 13 - Feb 17, 2009: Review Author Feedback and Discuss with Reviewers

You should be able to see the Author Rebuttal/Feedback to the reviews.You can start a discussion using the discussion board (see Phase 3: Discussion Phase) with the reviewers of the papers, especially if the rebuttal is responding to a specific review.  While, the reviews cannot be changed, you can take the input from the reviewers at this phase to help in your proposed decision and the consolidation report that you have drafted (or will!). Make sure to be clear about how the rebuttal was taken into account in the consolidation report.

Remember, all consolidation reports are considered a draft and all decisions are just proposed and will be finalized at the AC Meeting.

To See Consolidates Scores on the AC website:

To allow you to see the reviewer scores on the consolidation page, in the Area Chair “Consolidation Report” page, you can customize the page by selecting “Customize The View”->”Edit Review Columns”.  Select Q2 (“Overall Rating”) and click on “OK” to see the updated table. 

Phase 6. Feb 13 - Feb 20, 2009: Phase 6: Prepare for AC Meeting. Discuss with AC buddies, Propose decisions on paper. 

6.1 Feb. 13-20: Working with Buddies

We have enabled the system so you can now see the papers of your buddies (your buddy is listed in the table at the end of this email).  Go to “Consolidation Report” from your console and at top you will see

Show Assigned Papers | Show Buddy Papers | ..

Click on “Show Buddy Papers” and it will show all the papers that your buddy was assigned.  There should not be any conflict with any of the papers your buddy has. In the buddy paper list, if you don’t see your buddy AC’s name in the page, you can verify by clicking on any of the “View” link next to “Consolidation Report”.

Now here is a bit about the process.

Our Area Chair Committee is divided into Four Panels and within each panel, each AC is paired up with another AC.  This has been done in the past, so it is NOT new for the old-timers in this group.  The purpose of this is to have ACs work as a pair to make sure all decisions are made with some discussion with another person and to ensure all issues are dealt with fairly and appropriately.  Consider this as an opportunity for someone else to look over all your decisions and look at the reviews and the author rebuttal to ensure all was fair. 

Each buddy pair will be required to

1. Make Accept/Reject Decision Recommendations for each of their paper
   
2. Of the accepted papers, make a Recommendation for at least 6 papers as an ORAL.  These will be discussed in PANELs at the AC Meeting.
   
3. Of the accepted papers, make a Recommendation for at least 1 paper as potential AWARD papers (if more than 1, we need a rank-ordered list).  These will be forwarded to the Awards Committee, after some discussion within a panel.

Do note that buddies have ONLY READ access to the buddy papers.  The Assigned AC is the only one who can edit consolidation report and make decision recommendation and therefore it is important that both the AC discuss these and the lead AC consolidate.

So here is what to do.

1. Look at the reviews/rebuttals for each paper you have and then send a note to your AC buddy asking them to look at the papers where you would appreciate some help or at-least a look over. If you ask a specific question, like “can you see paper #XXX’s review #2 and see if you agree” OR “based on reviews of paper #YYY, I am considering a recommendation of Poster” or “paper #ZZZ is the best paper in my stack, do you agree?”
2. Discuss these papers with your buddy and write your consolidation report.
3. Make a recommendation of a decision on this paper and when consolidation report is done, “Check box if report is FINAL” and the PC Chairs will be informed on our consoles. 

Remember, you should try to do as much of this before the AC meeting and the first part of the AC meeting will also be dedicated to finishing this phase.

Here are the buddy pairs with their EMAILs (in PDF)  

6.2 Feb. 13-20: Writing Preliminary Consolidation Reports

Here is what a consolidation reports should contain.

• a summary (or highlights) of what the reviewers found in the paper that they liked/disliked
• a summary of discussions with the reviewers and your buddy (without names of course!) as the discussions influenced your evaluation.
• if a variance in reviews, explain you came to your evaluation.
• if decision is to overrule one of the reviewers, please provide as much details and also discuss with your buddy. The reviewers have looked at the paper longer then you (hopefully) so their evaluation should count and if dismissed, we need to have reasons for it. 

Phase 7. Feb 21 - Feb 22, 2009: AC Meeting.

• AC Meeting

Still need help?

• Look at the Area Chair Guidelines and FAQs.
• Contact us at cvpr09-pc-chairs@googlegroups.com
• For system-related problems (e.g., login problems or bandwidth issues), please contact cmt@microsoft.com (and mention the issue is CVPR09-related).

This post is dedicated to provide detailed guidelines to Area Chairs of CVPR 2009 as to how the process will unfold and what are their responsibilities.  Area Chairs should also see

• The Reviewer Guidelines as they have information about what is expected of reviewers, information about ethics of reviewing and best practices.  
• The AC Instructions (Instructions of how to use the reviewing site)
• The AC FAQ (Some of the recently asked questions and their answers)
• Google Calendar  (XML, ICAL, HTML) that you can subscribe to and see the schedule.

Role of Area Chairs.

The ACs help identify suitable reviewers, assign reviewers to papers, and evaluate the reviews as well as the authors’ responses. For CVPR09, our goal is to provide the highest quality reviews to authors and for this we hope to limit the number of papers assigned per AC to about 30-35 . This will allow each AC to play a more active role in ensuring higher quality and more timely reviews, and in initiating discussions amongst reviewers (anonymously), and in looking at authors’ responses to the reviews via the online rebuttal process. At the AC meeting in February 2009, ACs will work with other ACs (in pairs and then in panels) to decide which papers to accept and write consolidation reports to justify the decisions. Needless to say this is a very important part of the Academic process of Peer Reviewing and has a direct impact on quality of work in Computer Vision. All of the Area Chairs for CVPR 2009 are committed to doing this task in a timely, professional and ethical manner.

Best Practices of BEING an AC

We just want to remind all ACs of some of the basic (and very obvious) practices of being an AC.  

• You have a strong influence on a decision on a paper.  Take it seriously and be fair.
• DO NOT talk to any other AC about papers assigned to you until you are told about who is on your panel and as buddy.  There maybe many people who are AC who maybe conflicted with the paper you are an Area Chair for, and best if they don’t know that.  We will provide some details on how to do this as we get closer to the AC meeting. 
• The Program Co-Chairs did not submit any papers, so they are not in direct conflict with any papers and they know all authors of all papers and also all reviewers/AC. However, they will NOT be involved with any decisions of paper from institutions they are in conflict with. So you can always send them a note about any paper and if one of them is conflicted, another one will respond to you without including the conflicted PC Chair. 
• DO NOT talk to any other AC about your own paper (the paper you are an author on) or a paper you have some conflict with, during this whole process, unless that AC and you are already collaborators and both of you have a already defined conflict.  Do not lobby for your paper, hand out copies to other AC and show results or discuss your paper.  
• Advocate for papers your like, DO not kill papers you don’t like, unless of course you see a technical detail that is wrong and that would be un-acceptable to accept. Convince others to see your point of view.
• Be professional and willing to listen to other reviewers and ACs. DO not give in to undue influence from anyone.
• Do remember, (we assume with just cause, knowing the high quality reviewers we have) that the reviewers have spend time reading the paper. It is not fair to dismiss any review without looking at other reviews and reading the paper yourself to come of a evaluation. Outright rejecting a review is not fair.  Please work hard to make sure that if you are over ruling a review of someone that you have some serious grounds to do so (just saying “this reviewer is plain wrong” without justifying is not something we want!). 
• Senior members of the committee, guide and advise the younger and first-time members.  First-time members, please feel free to ask for help as needed. 

Make Sure You Assigned the Papers Appropriate for YOU.

The Program Chairs are going to assign papers to you based on (a) papers from authors not in conflict with you (see conflict of interest in Reviewer Guidelines), (b) subject areas you chose when you registered on the review site, and (c) subject areas, the authors chose, when they registered the paper. As can be imagined, this will NOT be perfect as primarily as some authors have differing takes on some subject areas then we do and also, sometime the Program Chairs have to do some load-balancing.  

Area Chairs should look through all the papers assigned to them and ensure that they are (1) not conflicted and (2) are knowledgeable in the subject area. Please look at the PDF of the papers in addition to the title/abstract of the paper to ensure this.  You should look at these papers to identify reviewers too.

Assigning Reviewers.

Remember, we want you to recommend 5 reviewer for each paper assigned to you. We will try to find 3 of these for the paper, trying to balance reviewer load and your choices. Here are some suggestions on how to proceed.  Remember, spend good time doing this, as it will really help with your role as an AC.  Dedicate parts of day to do this.  It can take anywhere from 4-6 minutes a paper do this, if you look at the paper and other info to help make a decision. Considering a max load of 35 papers, you can see, that this may take some time.  Trust us, it will be worth it. See Review Instructions for details of how to use the system to assign reviewers. 

• You have been chosen as an Area Chair because of your specific expertise in certain area of Vision and also you general expertise in Computer Vision. When you look at a paper, you may just come up with a name or three of ideal reviewers.  Consider these people as the reviewers for this, but do also consider things like, (a) are they current in the field and (b) will they do a good job in a timely manner.
• Look at the paper, especially the Introduction, Related Work, and Citations at the end.  See who they refer to and whose work they are building on.  On the cited people would be an ideal reviewers
• Do remember, that the system will give you some recommendation, based on simple heuristic by matching subject areas. This will provide a good starting point, but will not be perfect as some folks are not that good at choosing keywords (authors and reviewers).   
• Usually, we suggest that you choose a reviewer you know and trust. But vision community us large (how else would we get 1450+ submissions). If the system recommends someone you do not know, look for their webpage or look for their papers on on ACM or IEEE digital libraries or Google Scholar and the like.
• A new feature has been added to the system to allow you to add a new reviewer to a paper.  New reviewer, as an a reviewer that we did already get registered into the system before the deadline. Please use this carefully as we do not want to have a lots of outstanding review as reviewers were added in late in the process. See Review Instructions for info on how to add a reviewer on the fly.
• Again, spend good time doing this.

Keeping Track of Reviews and Following up with Reviewers.

After you have ranked reviewers, the Program Chairs, with the review system’s help will distribute papers to the reviewers. During this phase, we recommend that you log into the system to check if any reviews are being entered.  Do remember that the reviewers DO NOT KNOW your identity, so best if you do not contact them and/or identify yourself.  Yet another form “anonymity” in the review process, as we don’t want the reviewers to be unduly influenced by anyone.  If reviews are showing up, look at them and see if (a) the reviewer has added sufficient detail and (b) has appropriate tone in the review, and most importantly (e) providing a knowledgeable review. Contact the reviewers via the review system to get them to clarify issues from the review.  Doing this during the process may prevent you from doing it all at the deadline of when reviews are due.

After the review deadline has passed, check if all reviews are in.  If not, send them a note and nicely DEMAND that they finish the reviews soon.  The system will also send reminders to them.

Reaching Consensus

Coming Soon

How to best use the Rebuttal Process?

Coming Soon

Working in AC Pairs (”buddies”).

Coming Soon

Working in AC Panels

Dos and Don’ts at the AC Meeeting

Making Decisions

Consolidation Reports

Area Chair Consolidation Reports are the MOST CRUCIAL aspect of the review process of CVPR.  This is where the Area Chair justifies his/her recommendation to accept/reject a paper. The Program Chairs will read all Consolidation Reports and we have committed ourselves to ensure that these reports are representative of a good quality review process, which we are guaranteeing to the Vision community resting on the Area Chair’s shoulders.  These reports should highlight why the decision was reached. If all reviewers agree on a paper, then this consolidation report can be simple, but feel free to encourage authors and provide constructive feedback.  If there is even a slight disagreement on the reviews, it is your job to clarify why how the disagreement was resolved. Again, just saying “reviewer 1 was wrong” without any reasons is not acceptable.  Explain why.  For the most divergent reviews, explain the process you went through to reach a decision and if you and others read the paper, try to add some details from your and others reviews with the consolidation reports.  Over ruling all three reviewers to make a decision on a paper is not acceptable to us.  If you strongly feel that the paper deserves that make sure to discuss it in detail with your AC Buddy and also in your AC Panel and try to solicit an additional review.  In such cases, (a) you should read the paper completely and (b) provide a detailed consolidation report.

Conflicts with Papers Assigned to Area Chairs.

• I recognize a piece of work (ie I know the authors) should I recuse myself from the paper?  They are not collaborators and I have no financial ties or other connections with them. My feeling is that I can be impartial and handle the review but it is your call.
  • If you think you can do the review impartially and fairly, and there is no direct conflict, we think you should review this paper, as we feel you are best qualified for it.  Thanks for letting us know.
• I have been assigned a paper that might be pretty close to things I am working on at present (I don’t know for sure since I didn’t want to look at the full paper). I would like to decline handling that paper, if possible, on grounds of potential conflict of interest.
  • Thanks for letting us know.  Yes, this does qualify as a conflict of interest so we will assign to some other Area Chair. 
• I see a paper that I know is authored by a recent collaborator of mine (collaborated in last 3 years). I am reasonably confident I can still render a fair and an impartial review. But wanted to let you know.
  • This appears to be a conflict.  While, we do trust you to be fair and impartial, it is best to have someone else take over this paper.  Thanks for letting us know.  We will swap it.
• I have a paper assigned to me that appears to be very similar to one that I have just co-authored for CVPR 2009. I can be still objective in review of this paper, irrespective of my submission.
• Thanks for telling us.  We feel you are the expert in the area and trust you will do a good job. If you feel you cannot, then we will re-assign to another area chair.

Assigning Reviewers.

• What should I do with a paper with the names of the authors explicitly mentioned ?
  • Papers with authors names explicitly violate the anonymity requirement of CVPR and and will be rejected. Let the Program Chairs know the paper id. Thanks.
• What should I do with a paper that did not exactly follow the required format?
  • If the paper is still 2 column style CVPR format (missing the line numbers, paper id on every page, etc.) and not longer then 8 pages (font not too small, margins ok, etc.),  then it is fine to assign reviewers to (We don’t want to be too rigid).
• What about papers that over 8 pages long?
  • Overall, our hard-line policy is (as the authors were warned!) that the paper will be rejected. If you see a paper with just a citation or 1-2 lines on page 9, then we are letting it go. But if the paper is grossly over onto page 9, then it is a “administrative” reject.
• Is there a race to assign reviewers? If I do not assign reviewers by Dec 4, will I loose all the good reviewers for papers assigned to me?
  • No, there is NO race.  You are choosing (recommending) 5+ reviewers for each paper and on Dec 14, we will optimize the assignments based on (a) reviewer load, (b) your choice/rank of a reviewer, and (c) subject area match to reviewer. We hope this way, good reviewers per your choice will be assigned the paper you are an AC of.  However, we do not want to see all of waiting until Dec 8 and after to assign papers.
• I have scanned a paper and it seems very weak and will be rejected, what should I do?
• Unfortunately, if the paper has been submitted and does not meet a direct criteria for an administrative reject (non-anonymous, too long, dual-submission, etc.) then the paper has to be reviewed.  Give the paper a fair chance and have it reviewed.  We agree that sometimes this is a waste of reviewing resources, but we feel the authors deserve the best from us.

Assigning New Reviewers

• When should I decide to assign a new reviewer that is not in the database of reviewers?
  • After you scan a paper to see what is the content, some obvious names of reviewers will come to your mind.  Look for them in the reviewer lists.  All reviewers should be visible to you (there are over 700 of them, so it is a bit slow).  They are sorted by “subject area relevance” (matching their chosen subject areas to authors’ subject areas for their paper). If you do not see the person you wish to review the paper, then consider adding them as an additional reviewer.  Consider however that (a) this person may already have been asked to review for CVPR09 and may have declined and (b) this person may not have the time to review for this time.  So DO NOT consider that the new reviewer will agree to do this review.  Add an extra reviewer as a back-up for this one. 

Area Chairs Meeting

• When and where is the Area Chairs’ Meeting?
  • AC Meeting will be held on the GA Tech Campus, Feb 21-22 (Sat/Sun), 2009. All AC are expected to arrive on Feb 20 (Fri), 2009, and be there until the session ends on the evening of Feb 22 (Sun), which will then be followed by a dinner. All ACs are invited to stay an extra day, Feb 23 (Mon), 2009 to participate in an informal workshop on computer vision related topics.
• When should we make reservations for the AC meeting?
  • Soon you will be getting instructions from General Chair Jim Rehg and Finance Chair Terry Boult about how to plan for your travel and how you will be reimbursed.

Thank you for agreeing to review a paper for IEEE CVPR 2009. Your reviews have a direct and important impact on the quality of the an important conference in computer vision. Your reviews also help the computer vision community as a whole to improve the quality of its research. Please read through the rest of this document that provide details on what is expected of you as a member of the Papers Reviewing Committee for CVPR 2009.  If you have any questions, see Reviewer FAQs

Timely Reviews

The deadline for completed reviews is Jan 29, 2009 . The papers will be assigned to you by December 15, 2008. So you have almost a month and half to do these reviews, please do not leave them for the last few days near the deadline. The Area Chairs have a lot of work to do after the reviews are in. Adhering to this deadline is extremely important.

As soon as you get your reviewing assignment, please go through all the papers to make sure that (a) there is NO obvious conflict with you (as in it is your recent collaborator from a different institution) and (b) you are qualified to review the paper assigned. If these issues arise, please respond right away using the system. Contact us also if you find a paper that violates any of the paper submission guidelines described in here.

We will once again be offering an author rebuttal process this year preceding the Area Chair Committee meeting. Area Chairs will also follow up with you to get clarifications on reviews and as needed, seek consensus on diverging reviews.

What to Look For

Look for what’s good or stimulating in the paper. Minor flaws can be corrected and shouldn’t be a reason to reject a paper. Each paper that is accepted should, however, be technically sound and make a substantial contribution to the field. Please familiarize yourself with the information in the Call for Submissions.

Blind Reviews

Blind reviewing is an essential part of CVPR reviewing. Authors were asked to take reasonable efforts to hide their identities, including not listing their names or affiliations and omitting acknowledgments. This information will of course be included in the published version. Reviewers should also make all efforts to keep their identity invisible to the authors. Don’t say, “you should have cited my paper from 2006!”

Be Specific

Please be specific and detailed in your reviews. In the discussion of related work and references, simply saying “this is well known” or “this has been common practice in the industry for years” is not sufficient: cite specific publications or public disclosures of techniques! The Explanation section is easily the most important of the review. Your discussion, sometimes more than your score, will help the Area Chairs decide which papers to accept, so please be thorough. Your reviews will be returned to the authors, so you should include any specific feedback on ways the authors can improve their papers. For more suggestions on writing your reviews, read the section below on Writing Technical Reviews.

When You’re Done

When you have finished with your review, you should destroy any paper manuscript and/or supporting material you received. See the Ethics guidelines below.

Writing Technical Reviews

Here are some recommendations that may help you as you do this very valuable task.

In many professions, people give back to their community by doing volunteer work. In technical fields, we volunteer our time by reviewing papers that are written by other researchers in our field. We recommend that you approach your reviews in this spirit of volunteerism. Sure, your reviews make you a gatekeeper in helping decide which papers are ready for publication. Just as important, however, is to provide feedback to the authors so that they may improve their work. Try to write your review in a way that the authors can benefit from your review.

We suggest reading a paper and then thinking about it over the course of several days before I write my review. “Living” with a paper for a few days gives you time to make thoughtful decisions about it. This is the best way to come up with helpful suggestions for improving the paper. To do this, you need to carve out some time in your day to think about the paper that you are reviewing.

The tone of your review is important. A harshly written review will be disregarded by the authors, regardless of whether your criticisms are true. If you take care, it is always possible to word your review diplomatically while staying true to your thoughts about the paper. Put yourself in the mindset of writing to someone you wish to help, such as a respected colleague who wants your opinion on a concept or a project.

Here are some specific issues to keep in mind as you write your reviews:

• Short reviews are unhelpful to the authors and to other reviewers. If you have agreed to review a paper, you should take enough time to write a thoughtful and detailed review.
• Be specific when you suggest that the writing needs to be improved. If there is a particular section that is unclear, point it out and give suggestions for how it can be clarified.
• Don’t give away your identity by asking the authors to cite several of your own papers.
• Don’t just suggest your past work as possible citations, as it becomes obvious that you are asking for citations to your own and results in the authors just ignoring your review as someone who is interested in getting more cites to their own work.
• If you don’t think the paper is right for the CVPR program, suggest other publication possibilities (journals, conferences, workshops) that would be a better match for the paper.
• Avoid referring to the authors by using the phrase “you” or “the authors.” These phrases should be replaced by “the paper.” Directly talking about the authors can be perceived as being confrontational, even though you do not mean it this way.

Be generous about giving the authors new ideas for how they can improve their work. Your suggestions may be very specific (for example, “this numerical solver would be better for your application”) or may be more general in nature. You might suggest a new dataset that could be tried, or a new application area that might benefit from their tool. You may tell them how their idea can be generalized beyond what they have already considered.

A thoughtful review not only benefits the authors, but may well benefit you, too. Remember that your reviews are read by other reviewers and especially the Area Chairs, in addition to the authors. Being a helpful reviewer will generate good will toward you in the research community.

Ethics for Reviewing Papers

1. Protect Ideas

As a reviewer for CVPR, you have the responsibility to protect the confidentiality of the ideas represented in the papers you review. CVPR submissions are by their very nature not published documents. The work is considered new or proprietary by the authors; otherwise they would not have submitted it.

Of course, their intent is to ultimately publish to the world, but most of the submitted papers will not appear in the CVPR proceedings. Thus, it is likely that the paper you have in your hands will be refined further and submitted to some other journal or conference, or even to CVPR next year. Sometimes the work is still considered confidential by the author’s employers. These organizations do not consider sending a paper to CVPR for review to constitute a public disclosure. Protection of the ideas in the papers you receive means:

• Do not show the paper to anyone else, including colleagues or students, unless you have asked them to write a review, or to help with your review.
• Do not show any results or videos/images or any of the supplementary material to non-reviewers.
• Do not use ideas from papers you review to develop new ones.
• After the review process, destroy all copies of papers and videos that are not returned to the senior reviewer and erase any implementations you have written to evaluate the ideas in the papers, as well as any results of those implementations.

2. Avoid Conflict of Interest

As a reviewer of a CVPR paper, you have a certain power over the reviewing process. It is important for you to avoid any conflict of interest. Even though you would, of course, act impartially on any paper, there should be absolutely no question about the impartiality of review. Thus, if you are assigned a paper where your review would create a possible conflict of interest, you should return the paper and not submit a review. Conflicts of interest include (but are not limited to) situations in which:

• You work at the same institution as one of the authors.
• You have been directly involved in the work and will be receiving credit in some way. If you’re a member of the author’s thesis committee, and the paper is about his or her thesis work, then you were involved.
• You suspect that others might see a conflict of interest in your involvement. For example, even though Microsoft Research in Seattle and Beijing are in some ways more distant than Berkeley and MIT, there is likely to be a perception that they are “both Microsoft,” so folks from one should not review papers from the other.
• You have collaborated with one of the authors in the past three years (more or less). Collaboration is usually defined as having written a paper or grant proposal together, although you should use your judgment.
• You were the MS/PhD advisor of one of the authors or the MS/PhD advisee of one of the authors. Funding agencies typically consider advisees to represent a lifetime conflict of interest. CVPR has traditionally been more flexible than this, but you should think carefully before reviewing a paper you know to be written by a former advisee.

The blind reviewing process will help hide the authorship of many papers, and senior reviewers will try hard to avoid conflicts. But if you recognize the work or the author and feel it could present a conflict of interest, send the paper back to the senior reviewer as soon as possible so he or she can find someone else to review it.

3. Be Serious

The paper publishing business in CVPR is very serious indeed: careers and reputations hinge on publishing in the proceedings, academic tenure decisions are based on the proceedings, and patent infringement cases have discussed whether something was considered novel enough to publish in the proceedings. This does not mean that we cannot have any fun in the paper sessions. But it does mean that we have a responsibility to be serious in the reviewing process. You should make an effort to do a good review. This is obvious. But one of the complaints we have heard about the CVPR review process is that some reviews can be so sketchy that it looks like the reviewer did not even seem to take the time to read the paper carefully. A casual or flippant review of a paper that the author has seriously submitted is not appropriate. In the long run, casual reviewing is a most damaging attack on the CVPR conference. There is no dishonor in being too busy to do a good review, or to realize that you have over-committed yourself and cannot review all the papers you agreed to review. But it is a big mistake to take on too much, and then not back out early enough to allow recovery. If you cannot do a decent job, give the paper back and say so. But please, do it early so that the the Area Chairs and Program Chairs have time to select another reviewer before the deadline.

4. Be Professional

Belittling or sarcastic comments may help display one’s wit, but they are unnecessary in the reviewing process. The most valuable comments in a review are those that help the authors understand the shortcomings of their work and how they might improve it. If you intensely dislike a paper, justify it constructively and still provide feedback to the authors.  If you give a paper a low score, it is essential that you justify the reason for the score in detail.  Just saying “I do not like this approach because I have 10+ years of experience in this area” is NOT constructive. You need to share your professional opinion. Do not just cite your own past work, it maybe true, but it seems like you just want more cites to your own work and may result in the authors just ignoring your review (and maybe the Area Chair too!).

5. In Summary

Adherence to ethics makes the whole reviewing process more complicated and sometimes less efficient. But convenience, efficiency, and expediency are not good reasons to contravene ethics. It is precisely at those times when it would be easier or more efficient to bend the rules that it is most important to do the right thing. Ultimately, spending that energy and time is an investment in the long-term health of the technical-paper sessions, the conference, and the community of computer vision researchers.

Based on Specific Documents Created for SIGGRAPH 2008 by Greg Turk (used here with permission). UPDATE and Modified by Irfan Essa.

• http://www.siggraph.org/s2008/submissions/juried/papers/instructions.php
• http://www.siggraph.org/s2008/submissions/juried/papers/review_writing.php
• http://www.siggraph.org/s2008/submissions/juried/papers/ethics.php

A few other sources of reviewer guidelines from ACM and IEEE were also considered.